Ip-based interconnection of switches with a logical chassis

ABSTRACT

One embodiment of the present invention provides a switch. The switch includes a logical channel apparatus and a tunnel apparatus. The logical channel apparatus associates a logical channel identifier of a logical channel with the switch and assigns an Internet Protocol (IP) address as switch identifier of the switch. The logical channel includes a plurality of member switches and the switch is a member switch of the logical channel. The IP address uniquely identifies the switch in the logical channel. The tunnel apparatus establishes a tunnel with a remote switch in the logical channel. An inter-switch packet from the switch is encapsulated in a tunnel header associated with the tunnel.

RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.62/219,572, Attorney Docket Number BRCD-3293.0.1.US.PSP, titled“IP-BASED INTERCONNECTION OF SWITCHES WITH A LOGICAL CHASSIS,” byinventors Phanidhar Koganti and Suresh Vobbilisetty, filed 16 Sep. 2015,the disclosure of which is incorporated by reference herein.

BACKGROUND

Field

This disclosure relates to communication networks. More specifically,the present disclosure relates to a method for a constructing a scalableswitching system.

Related Art

The exponential growth of the Internet has made it a popular deliverymedium for a variety of applications running on physical and virtualdevices. Such applications have brought with them an increasing demandfor bandwidth. As a result, equipment vendors race to build larger andfaster switches with versatile capabilities, such as networkvirtualization and multi-tenancy, to accommodate diverse network demandsefficiently. However, the size of a switch cannot grow infinitely. It islimited by physical space, power consumption, and design complexity, toname a few factors. Furthermore, switches with higher capability areusually more complex and expensive. More importantly, because an overlylarge and complex system often does not provide economy of scale, simplyincreasing the size and capability of a switch may prove economicallyunviable due to the increased per-port cost.

One way to increase the throughput of a switch system is to use switchstacking. In switch stacking, multiple smaller-scale, identical switchesare interconnected in a special pattern to form a larger logical switch.However, switch stacking requires careful configuration of the ports andinter-switch links. The amount of required manual configuration becomesprohibitively complex and tedious when the stack reaches a certain size,which precludes switch stacking from being a practical option inbuilding a large-scale switching system. Furthermore, a system based onstacked switches often has topology limitations which restrict thescalability of the system due to bandwidth considerations.

A flexible way to improve the scalability of a switch system is to buildan interconnection of switches that share a single logical chassis (alsoreferred to as “fabric switch”). A fabric switch is a collection ofindividual member switches. These member switches form a network ofinterconnected switches that can have an arbitrary number of ports andan arbitrary topology. As demands grow, customers can adopt a “pay asyou grow” approach to scale up the capacity of the fabric switch.

While a fabric switch brings desirable features, some issues remainunsolved in efficient formation and data transportation of a scalablefabric switch.

SUMMARY

One embodiment of the present invention provides a switch. The switchincludes a logical chassis apparatus and a tunnel apparatus. The logicalchassis apparatus associates a logical chassis identifier of a logicalchassis with the switch and assigns an Internet Protocol (IP) address asswitch identifier of the switch. The logical chassis includes aplurality of member switches and the switch is a member switch of thelogical chassis. The IP address uniquely identifies the switch in thelogical chassis. The tunnel apparatus establishes a tunnel with a remoteswitch in the logical chassis. An inter-switch packet from the switch isencapsulated in a tunnel header associated with the tunnel.

In a variation on this embodiment, the logical chassis apparatusmaintains a mapped identifier assigned to the switch. The mappedidentifier is an index for the switch in the logical chassis.

In a further variation, a port of the switch is identified by a portidentifier uniquely identifying the port in the logical chassis. Thisport identifier includes the mapped identifier.

In a variation on this embodiment, the logical chassis apparatusdetermines adjacency in the logical chassis by running a routingprotocol.

In a variation on this embodiment, the logical chassis apparatusoperates as the logical chassis as a single manageable entity forprovisioning, control, or both.

In a further variation, the logical chassis apparatus manages thelogical chassis based on one or more of: a command line interface (CLI),a Network Configuration Protocol (NETCONF), and RESTCONF.

In a variation on this embodiment, the switch is in a logical unit,which is a building unit of a logical chassis.

In a further variation, the logical unit includes a second switch, andthe switch and the second switch operate as tunnel end points for thetunnel.

In a variation on this embodiment, the switch also includes a logicaltunnel end point apparatus, which operates the logical chassis as an endpoint for an external tunnel. The other end point of the external tunnelis outside of the logical chassis.

In a variation on this embodiment, the switch also includes a linkaggregation apparatus, which identifies a plurality of links coupled toa same neighbor switch and operates the identified links as a linkaggregation group. The links in the link aggregation group operate as asingle logical link.

In a variation on this embodiment, the switch is in a software definednetwork and receives configuration information in an instruction from acontroller of the software defined network.

In a variation on this embodiment, the logical chassis apparatusestablishes a point-to-point connection with a neighbor switch using anunnumbered interface based on the IP address.

In a variation on this embodiment, the logical chassis apparatusdiscovers a neighbor switch based on a link discovery protocol.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A illustrates an exemplary Internet-Protocol-based (IP-based)fabric switch, in accordance with an embodiment of the presentinvention.

FIG. 1B illustrates an exemplary console for configuring an IP-basedfabric switch, in accordance with an embodiment of the presentinvention.

FIG. 1C illustrates an exemplary configuration database for an IP-basedfabric switch, in accordance with an embodiment of the presentinvention.

FIG. 1D illustrates exemplary logical units in an IP-based fabricswitch, in accordance with an embodiment of the present invention.

FIG. 2A illustrates exemplary inter-switch tunnels in an IP-based fabricswitch, in accordance with an embodiment of the present invention.

FIG. 2B illustrates an exemplary tunnel encapsulation header for anIP-based fabric switch, in accordance with an embodiment of the presentinvention.

FIG. 3A illustrates an exemplary IP-based fabric switch participating ina software defined network, in accordance with an embodiment of thepresent invention.

FIG. 3B illustrates exemplary Fibre Channel (FC) gateways in an IP-basedfabric switch, in accordance with an embodiment of the presentinvention.

FIG. 4 presents a flowchart illustrating the fabric-formation process ofa member switch in an IP-based fabric switch, in accordance with anembodiment of the present invention.

FIG. 5 illustrates an exemplary virtual link aggregation group in anIP-based fabric switch, in accordance with an embodiment of the presentinvention.

FIG. 6 illustrates an exemplary member switch in an IP-based fabricswitch, in accordance with an embodiment of the present invention.

In the figures, like reference numerals refer to the same figureelements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the invention, and is provided in the context ofa particular application and its requirements. Various modifications tothe disclosed embodiments will be readily apparent to those skilled inthe art, and the general principles defined herein may be applied toother embodiments and applications without departing from the spirit andscope of the present invention. Thus, the present invention is notlimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the claims.

Overview

In embodiments of the present invention, the problem of building aversatile, cost-effective, and scalable switching system is solved byforming a topology agnostic fabric switch based on an internal (orunderlay) layer-3 protocol. This internal layer-3 protocol operateswithin the fabric switch and may not advertise routes within the fabricswitch outside. A respective switch of the fabric switch can be referredto as a member switch. One can form a large-scale switch using a numberof smaller physical switches. In some embodiments, this fabric switchcan appear as a single logical entity in the provisioning and controlplane. This allows a user to provide configuration information to amember switch, which, in turn, propagates the configuration informationto other member switches. In this way, a respective member switch canlocally apply the configuration information.

In some embodiments, the control plane running on a respective memberswitch allows any number of switches to be connected in an arbitrarytopology without requiring tedious manual configuration of the ports andlinks. This feature makes it possible to use many smaller, inexpensiveswitches to construct a large network, which can operate as a singleswitch in the data plane as well. When a member switch of such a fabricswitch learns a media access control (MAC) address of an end device(e.g., via layer-2 MAC address learning), the member switch generates anotification message, includes the learned MAC address in the payload ofthe notification message, and sends the notification message to allother member switches of the fabric switch. In this way, a learned MACaddress is shared among a respective member switch of the fabric switch.

It should be noted that a fabric switch is not the same as conventionalswitch stacking. In switch stacking, multiple switches areinterconnected at a common location (often within the same rack), basedon a particular topology, and manually configured in a particular way.These stacked switches typically share a common address, e.g., an IPaddress, so they can be addressed as a single switch externally.Furthermore, switch stacking requires a significant amount of manualconfiguration of the ports and inter-switch links. The need for manualconfiguration prohibits switch stacking from being a viable option inbuilding a large-scale switching system. The topology restrictionimposed by switch stacking also limits the number of switches that canbe stacked. This is because it is very difficult, if not impossible, todesign a stack topology that allows the overall switch bandwidth toscale adequately with the number of switch units.

In contrast, a fabric switch can include an arbitrary number of switcheswith individual addresses, can be based on an arbitrary topology, anddoes not require extensive manual configuration. The switches can residein the same location, or be distributed over different locations. Thesefeatures overcome the inherent limitations of switch stacking and makeit possible to build a large “switch farm,” which can be treated as asingle, logical switch. Due to the automatic configuration capabilitiesof the fabric switch, an individual physical switch can dynamically joinor leave the fabric switch without disrupting services to the rest ofthe network.

Furthermore, the automatic and dynamic configurability of the fabricswitch allows a network operator to build its switching system in adistributed and “pay-as-you-grow” fashion without sacrificingscalability. The fabric switch's ability to respond to changing networkconditions makes it an ideal solution in a virtual computingenvironment, where network loads often change with time.

It should also be noted that a fabric switch is distinct from a VLAN. Afabric switch can accommodate a plurality of VLANs. A VLAN is typicallyidentified by a VLAN tag. In contrast, the fabric switch is identifiedby a fabric identifier (e.g., a cluster identifier), which is assignedto the fabric switch. Since a fabric switch can represented as a logicalchassis, the fabric identifier can also be referred to as a logicalchassis identifier. A respective member switch of the fabric switch isassociated with the fabric identifier. In some embodiments, a fabricswitch identifier is pre-assigned to a member switch. As a result, whenthe switch joins a fabric switch, other member switches identifies theswitch to be a member switch of the fabric switch.

In this disclosure, the term “fabric switch” refers to a number ofinterconnected physical switches which form a single, scalable networkof switches. The member switches of the fabric switch can operate asindividual switches. The member switches of the fabric switch can alsooperate as a single switch in the provision and control plane, the dataplane, or both. “Fabric switch” should not be interpreted as limitingembodiments of the present invention to a plurality of switchesoperating as a single, logical switch.

Although the present disclosure is presented using examples based on anencapsulation protocol, embodiments of the present invention are notlimited to networks defined using one particular encapsulation protocolassociated with a particular Open System Interconnection Reference Model(OSI reference model) layer. For example, embodiments of the presentinvention can also be applied to a multi-protocol label switching (MPLS)network. In this disclosure, the term “encapsulation” is used in ageneric sense, and can refer to encapsulation in any networking layer,sub-layer, or a combination of networking layers.

The term “end device” can refer to any device external to a network(e.g., does not perform forwarding in that network). Examples of an enddevice include, but are not limited to, a physical or virtual machine, aconventional layer-2 switch, a layer-3 router, or any other type ofnetwork device. Additionally, an end device can be coupled to otherswitches or hosts further away from a layer-2 or layer-3 network. An enddevice can also be an aggregation point for a number of network devicesto enter the network. An end device hosting one or more virtual machinescan be referred to as a host machine. In this disclosure, the terms “enddevice” and “host machine” are used interchangeably.

The term “VLAN” is used in a generic sense, and can refer to anyvirtualized network. Any virtualized network comprising a segment ofphysical networking devices, software network resources, and networkfunctionality can be can be referred to as a “VLAN.” “VLAN” should notbe interpreted as limiting embodiments of the present invention tolayer-2 networks. “VLAN” can be replaced by other terminologiesreferring to a virtualized network or network segment, such as “VirtualPrivate Network (VPN),” “Virtual Private LAN Service (VPLS),” or “EasyVirtual Network (EVN).”

The term “packet” refers to a group of bits that can be transportedtogether across a network. “Packet” should not be interpreted aslimiting embodiments of the present invention to layer-3 networks.“Packet” can be replaced by other terminologies referring to a group ofbits, such as “frame,” “cell,” or “datagram.”

The term “switch” is used in a generic sense, and can refer to anystandalone or fabric switch operating in any network layer. “Switch” canbe a physical device or software running on a computing device. “Switch”should not be interpreted as limiting embodiments of the presentinvention to layer-2 networks. Any device that can forward traffic to anexternal device or another switch can be referred to as a “switch.”Examples of a “switch” include, but are not limited to, a layer-2switch, a layer-3 router, a TRILL RBridge, or a fabric switch comprisinga plurality of similar or heterogeneous smaller physical switches.

The term “edge port” refers to a port on a network which exchanges dataframes with a device outside of the network (i.e., an edge port is notused for exchanging data frames with another member switch of anetwork). The term “inter-switch port” refers to a port whichsends/receives data frames among member switches of the network. A linkbetween inter-switch ports is referred to as an “inter-switch link.” Theterms “interface” and “port” are used interchangeably.

The term “switch identifier” refers to a group of bits that can be usedto identify a switch. Examples of a switch identifier include, but arenot limited to, a media access control (MAC) address, an InternetProtocol (IP) address, an RBridge identifier, or a combination thereof.In this disclosure, “switch identifier” is used as a generic term, isnot limited to any bit format, and can refer to any format that canidentify a switch.

The term “tunnel” refers to a data communication where one or morenetworking protocols are encapsulated using another networking protocol.Although the present disclosure is presented using examples based on alayer-3 encapsulation of a layer-2 protocol, “tunnel” should not beinterpreted as limiting embodiments of the present invention to layer-2and layer-3 protocols. A “tunnel” can be established for and using anynetworking layer, sub-layer, or a combination of networking layers.

Network Architecture

FIG. 1A illustrates an exemplary IP-based fabric switch, in accordancewith an embodiment of the present invention. As illustrated in FIG. 1A,a fabric switch 100 includes member switches 101, 102, 103, 104, and105. Fabric switch 100 can be based on IP and a respective memberswitch, such as switch 105, can be an IP-capable switch, whichcalculates and maintains a local IP routing table (e.g., a routinginformation base or RIB), and is capable of forwarding packets based onits IP addresses. The routing table specifies routes within fabricswitch 100. To populate the IP routing table, a respective member switchuses a routing protocol (e.g., OSPF-based routing protocol). In someembodiments, one or more switches in fabric switch 100 can be virtualswitches (e.g., a software switch running on a computing device).Switches 101 and 104 are coupled to end devices 112 and 114,respectively.

Member switches in fabric switch 100 use edge ports to communicate withend devices and inter-switch ports to communicate with other memberswitches. For example, switch 101 is coupled to end device 101 via anedge port and to switches 102, 103, 104, and 105 via inter-switch ports.Communication between member switches via inter-switch ports can bebased on IP, and communication between an end device and a member switchvia an edge port can be based on Ethernet. For example, switch 104receives an Ethernet frame from end device 114 via an edge port. Switch104 then encapsulates the Ethernet frame in an IP header (e.g., alayer-3 tunnel header) and forwards the encapsulated packet to anothermember switch. It should be noted that the encapsulated packet can havean external Ethernet header for layer-2 forwarding.

A respective switch in fabric switch 100 is assigned a switchidentifier, such as an IP address (e.g., an IP v4 or IP v6 address). Auser (e.g., a network administrator) can assign the switch identifier toa respective member switch. For example, end device 112 can be anadministrator workstation and the user can assign a switch identifier ofa respective member switch from end device 112. A switch can also bedynamically assigned to a switch (e.g., using a Dynamic HostConfiguration Protocol (DHCP) server). In some embodiments, fromprovisioning perspective (e.g., assigning an IP address), end device 112views fabric switch as a single logical entity, such as a logicalchassis 110. A respective member switch can appear as an element inlogical chassis 110. As a result, the user can configure fabric switch100 from a single location (e.g., end device 112), and globalconfigurations (i.e., the configurations applicable to a respectiveswitch) can be automatically applied to a respective member switch. Enddevice 112 can manage logical chassis 110 as a single manageable entity.In some embodiments, end device 112 can use command line interface (CLI)of a switch or a management protocol to manage logical chassis 110.Examples of a management protocol include, but are not limited to,Network Configuration Protocol (NETCONF) and RESTCONF.

Furthermore, a respective member switch is assigned a mapped identifier,which can be a switch index within fabric switch 100. This mappedidentifier can also be locally generated in a switch based on the localswitch identifier. The mapped identifier can operate as a “shortened”identifier for a switch. In the example in FIG. 1A, fabric switch 100includes five member switches. A switch identifier for a respectiveswitch in fabric switch 100 can be an IP address, which is 32 bits longfor IP v4 or 128 bits long for IP v6. However, a number represented bythree bits (e.g., integers 0-4) can identify the member switches infabric switch 100. Hence, a three-bit long mapped identifier can be usedto represent the member switches of fabric switch 100. In someembodiments, the number of bits dedicated for a mapped identifier infabric switch 100 is determined based on the maximum number of memberswitches supported by fabric switch 100. For example, if fabric switch100 supports a maximum 64 member switches, mapped identifier for fabricswitch 100 should be at least six bits long.

In some embodiments, a port in fabric switch 100 is assigned a portidentifier, which uniquely identifies the port in fabric switch 100. Aport identifier in fabric switch 100 can be in a “mappedidentifier/chassis number (e.g., line card number)/port number” format.If the mapped identifier of switch 101 is “X,” and switch 101 has atleast three chassis, one of which includes at least 16 ports, a portidentifier of switch 101 can be “X/2/15.” This identifier represents thesixteenth port of the third chassis of switch 101. Similarly, if themapped identifier of switch 102 is “Y,” and switch 102 also has at leastthree chassis, each of which includes at least 16 ports, a portidentifier of switch 102 can be “Y/2/15.” In this way, the mappedidentifier in a port identifier distinguishes two ports having the samechassis and port number in fabric switch 100. If a switch is a “pizzabox” switch with a single chassis, the chassis number in a portidentifier can be “0.”

During operation, a respective member switch of fabric switch 100 uses alink discovery protocol via its inter-switch links to discover aneighbor switch. Examples of a link discovery protocol include, but arenot limited to, Link Layer Discovery Protocol (LLDP) and Brocade LinkDiscovery Protocol (BLDP). In some embodiments, an inter-switch link canbe modeled as a point-to-point unnumbered interface to avoid IP addressand/or mask configuration for a respective inter-switch link. Forexample, when respective IP addresses are assigned to switches 103 and104 as switch identifiers, the inter-switch communication betweenswitches 103 and 104 can be established as a point-to-pointcommunication channel between the corresponding interfaces using the IPaddresses. This allows auto discovery of neighbors in fabric switch 100without configuring an individual IP address for a respective interface.

In some embodiments, fabric switch 100 is assigned a fabric identifier,which uniquely identifies fabric switch 100. The fabric identifier isassigned to a respective switch of fabric switch 100 (e.g., the user canconfigure from end device 112). Upon discovering each other, switches101 and 103 determine that they have the same fabric identifier offabric switch 100 and belong to the same fabric switch. This allows amember switch of fabric switch 100 to automatically detect other memberswitches and form fabric switch 100.

In some embodiments, inter-switch links in fabric switch 100 supportsautomatic formation of link aggregations. Suppose that three linkscouple switches 101 and 103. As a result, switch 101 discovers switch103 via all three links, and vice versa. For example, switch 101 canreceive a link discovery message (e.g., LLDP Data Unit (LLDPDU))comprising the same switch identifier of switch 103 via the three links.Switch 101 then determines that switch 101 is coupled to switch 103 viathose three links. Similarly, switch 103 also determines that switch 103is coupled to switch 101 via three links. Switch 101 and 103 thenautomatically aggregate the links between them to form an inter-switchlink aggregation group 130.

If a link in link aggregation group 130 becomes unavailable, other linkscan continue to operate. However, if the number of links becomes one,virtual link aggregation group 130 can become an individual link. Iflink aggregation group 130 becomes unavailable (e.g., due to multiplelink failures or a node failure), switches 101 and 103 detect theunavailability, and generate and send a notification message notifyingother member switches regarding the unavailability (e.g., in the payloadof a notification message). Similarly, if the link between switches 104and 105 becomes unavailable, switches 104 and 105 detect theunavailability, and generate and send a notification message. Uponreceiving the notification message, other member switches run therouting protocol based on the updated adjacency to determine the updatedpaths.

In link aggregation group 130, the links can be coupled to one or morenetwork interface cards (NICs). For example, if a set of links arecoupled to a NIC in switch 101 are also coupled to a NIC in switch 103,the set of links form a link trunk 132. Link trunk 132 and individualnon-trunk link 134 between switches 101 and 103 then form linkaggregation group 130. It should be noted that a link aggregation groupcan include a combination of link trunks and individual links.

Upon forming the link aggregation groups for inter-switch links, arespective member switch in fabric switch 100 runs a routing protocol todiscover adjacency in fabric switch 100. Examples of a routing protocolinclude, but are not limited to, Open Shortest Path First (OSPF) basedrouting protocols, distance vector based routing protocols, and acombination thereof. This routing protocol discovers one or more pathsbetween the member switches. For example, switch 103 discovers thatswitch 105 is reachable via a path comprising switch 104, and can assignswitch 104 as the next-hop switch for switch 105. If a link aggregationgroup exists between a member switch pair, the adjacency can be formedover that link aggregation group. For example, adjacency betweenswitches 101 and 103 can be formed over link aggregation group 130.

In some embodiments, a respective switch of fabric switch 100 supportspriority-based flow control (PFC). In this way, during packet forwardingwithin fabric switch 100, a member switch can provide a uniform qualityof service (QoS) for the outer and inner layer-2 headers. Duringoperation, switch 104 receives an Ethernet frame from end device 114.Switch 104 identifies the priority value associated with PFC in theEthernet header and encapsulates the frame in an IP header and an outerEthernet header. Switch 104 maps the identified priority value in theouter Ethernet header (e.g., a one-to-one mapping) and forwards theencapsulated packet based on its destination. Upon receiving the packet,any other member switch applies priority-based flow control based on thepriority value in the outer Ethernet header to the packet.

In some embodiments, a member switch can share information with anothermember switch in fabric switch 100 based on a name service. Uponlearning a MAC address, a member switch includes the learned MAC addressin a payload of a name service notification message and sends themessage a respective other switches of fabric switch 100 via the nameservice. In this way, a respective member switch is aware of thelocations of the end devices coupled with fabric switch 100. In someembodiments, the payload format of the notification message is the sameregardless of the protocol based on which the name service. This allowsthe name service to be backward compatible. The name service can beimplemented based on a scale protocol, such as ZeroMQ and NanoMsg.

FIG. 1B illustrates an exemplary console for configuring an IP-basedfabric switch, in accordance with an embodiment of the presentinvention. In this example, switch 101 has a console 150. Upon accessing(e.g., from end device 112), switch 101 presents console 150 to theuser. In some embodiments, when the user accesses switch 101, console150 provides a command line interface shell 152 to the user. The usercan type commands to shell 152. Shell 152 can be the initial screenwhich appears when the user accesses switch 101. Suppose that fabricswitch 100 has a fabric identifier 142, which identifies fabric switch100 and is associated with a respective switch of fabric switch 100.Since fabric switch 100 can operate as a single logical chassis for theprovision and control plane, the user can provide a command to shell 152to gain access to fabric switch 100 as a logical chassis (e.g.,“fabric-switch fabric-id 142”). This allows the user to provision fabricswitch 100 as logical chassis 110.

The user then provides another command (e.g., “config terminal”) toshell 152 to gain access to a configuration terminal for fabric switch100 in shell 152. The user can use this configuration terminal toprovide global configuration associated with fabric switch 100 and localconfiguration associated with any member switch in fabric switch 100.For example, if switch 101 has a switch MAC address 144, the user canissue a command to the terminal to map MAC address 144 to a switchidentifier 146 (e.g., a switch IP address). Similarly, the user canissue another command to the terminal to map switch identifier 146 to amapped identifier 148. In some embodiments, switch 101 can generatemapped identifier 148 from switch identifier 146 (e.g., without a userconfiguration). Mapped identifier 148 can also be pre-assigned to switch101.

The user can issue a command to the terminal to create a VLAN 110. ThisVLAN 110 is created across fabric switch 100, and hence, is part of theglobal configuration. On the other hand, user can also issue a commandto configure a specific port of a specific switch in fabric switch 100.This port configuration is a local configuration for that switch. Insome embodiments, a port is identified by a port identifier, which canbe in a “mapped identifier/chassis number (e.g., line card number)/portnumber” format. For example, the user can configure a 10 GigabitEthernet port identified by port identifier “148/2/15.” Since switch 101is associated with mapped identifier 148, the port identifier indicatesthat the port is port number 16 in line card number 3 of switch 101. Theuser can add VLAN 110 to that specific port. This VLAN configuration ofthe port is a local configuration of switch 101.

FIG. 1C illustrates an exemplary configuration database for an IP-basedfabric switch, in accordance with an embodiment of the presentinvention. As illustrated in FIG. 1C, a member switch of fabric switch100 typically maintains two configuration tables that describe itsinstance: a fabric switch configuration database 180, and a defaultswitch configuration table 184. Configuration database 180 describes thefabric switch configuration when a switch is part of fabric switch 100.Default switch configuration table 184 describes the switch's defaultconfiguration. Configuration database 180 includes a globalconfiguration table (GT) 182, which includes a fabric switch identifier,such as fabric identifier 142 for fabric switch 100 (denoted asFABRIC_ID), and a VLAN list in fabric switch 100. Also included inconfiguration database 180 are a number of switch (or local)configuration tables (STs or LTs), such as STO, ST1, and STn. Each STincludes the corresponding member switch's MAC address and the switchidentifier, as well as the switch's interface details.

In some embodiments, when a switch joins fabric switch 100 for the firsttime, fabric switch 100 assigns a mapped identifier to the switch. Forexample, fabric switch 100 assigns a value of “0” to mapped identifier148 of switch 101 and stores in corresponding ST0. This mappedidentifier persists with switch 101, even if switch 101 leaves fabricswitch 100. When switch 101 joins fabric switch 100 again at a latertime, the same mapped identifier “0” is used by fabric switch 100 toretrieve previous configuration information for switch 101. This featurecan reduce the amount of configuration overhead in fabric switch 100.Also, the persistent mapped identifier allows fabric switch 100 to“recognize” a previously configured member switch 101 when it re-joinsfabric switch 100, since a dynamically assigned switch identifier canchange each time switch 101 joins and is configured by fabric switch100.

Default switch configuration table 184 has an entry for the mappedidentifier that points to the corresponding ST in configuration database180. Note that configuration database 180 is replicated and distributedto all switches in fabric switch 100. Default switch configuration table184 is local to a particular member switch.

The “IN_FABRIC” value in default switch configuration table 184indicates whether the member switch is part of a fabric switch. A switchis considered to be “in a fabric switch” when it is assigned one of theswitch identifiers by a fabric switch. When a switch is first connectedto fabric switch 100, fabric switch formation process allocates a newswitch identifier to the joining switch. In one embodiment, only theswitches directly connected to the new switch participate in the joinoperation.

Note that in the case where the global configuration database of ajoining switch is current and in sync with the global configurationdatabase of fabric switch 100 based on a comparison of the transactionidentifiers of the two databases (e.g., when a member switch istemporarily disconnected from fabric switch 100 and re-connected shortlyafterward), a trivial merge is performed. That is, the joining switchcan be connected to fabric switch 100, and no change or update to theglobal configuration database is required.

FIG. 1D illustrates exemplary logical units in an IP-based fabricswitch, in accordance with an embodiment of the present invention. Thelogical building block of fabric switch 100 can be a single switch or agroup of switches. The selection of a building block can be based on thecustomer requiring a switch-level high availability. To facilitate highavailability among member switches of fabric switch 100, a plurality ofswitches can form a logical unit, and a logical unit can operate as asingle logical member of fabric switch. In some embodiments, switches103 and 104 can form a logical unit 162, and switches 101 and 102 canform another logical unit 164. A fabric switch can include a combinationof logical units and standalone member switches. In the example in FIG.1D, fabric switch 100 includes logical units 162 and 164, and standalonemember switch 105. In some embodiments, standalone switch 105 canoperate as a logical unit.

A respective switch can be assigned a logical unit identifier, whichidentifies to which logical unit the switch belongs. For example,switches 103 and 104 can have the same logical unit identifier, whichidentifies logical unit 162. During operation, a switch discovers otherswitches with the same fabric identifier and logical identifier, andautomatically forms a logical unit with the discovered switches. Theswitches in a logical unit can operate in an active-standby mode (oneswitch remains active and others are in standby mode), or in anactive-active mode (a respective switch receives and forwards traffic).

Tunnel-Based Fabric Encapsulation

In some embodiments, a respective member switch forwards traffic infabric switch 100 based on an encapsulation header (e.g., a tunnelencapsulation header). FIG. 2A illustrates exemplary inter-switchtunnels in an IP-based fabric switch, in accordance with an embodimentof the present invention. Suppose that fabric switch 100 is coupled witha layer-3 core network 200 via switches 101 and 102. For forwardingtraffic within fabric switch 100, all member switches do not need toparticipate in the routing protocol of network 200 (e.g., the overlayrouting protocol). Switches 101 and 102 can participate in the routingprotocol of network 200 and operate as gateways. It should be noted thata respective member switch, including switches 101 and 102, participatein the internal (or underlay) routing protocol of fabric switch 100 forIP-based forwarding within fabric switch 100. The internal routingprotocol interfaces with network 200. The internal routing protocoladvertises the subnets associated with the overlay network as connectedroutes to network 200.

In some embodiments, inter-switch packet forwarding in fabric switch 100is based on tunnel encapsulation. Examples of a tunnel encapsulationprotocol include, but are not limited to, Virtual Extensible Local AreaNetwork (VXLAN), Generic Routing Encapsulation (GRE), and itsvariations, such as Network Virtualization using GRE (NVGRE) andopenvSwitch GRE. A respective switch of fabric switch 100 establishes atunnel with a respective other member switch (e.g., a full mesh oftunnels). For example, switch 103 establishes tunnels 212, 214, 216, and218 with switches 101, 102, 104, and 105 respectively. If a tunnel spansan inter-switch link aggregation, that tunnel can use all links in thelink aggregation to forward traffic. For example, tunnel 212 can use alllinks in link aggregation 130.

It should be noted that, even though inter-switch packet forwarding infabric switch 100 is based on a tunnel encapsulation header, fabricswitch 100 represents itself as a single logical tunnel end point (TEP)210 to external end devices. Logical tunnel end point 210 can beassociated with a virtual IP address. For example, if end device 204establishes a tunnel with fabric switch 100 via network 200, fabricswitch 100 represents itself as logical tunnel end point 210. If enddevice 204 sends a packet via the tunnel, end device 204 encapsulatesthe packet in a tunnel encapsulation header with the virtual IP addressas the egress address

One or more member switches of fabric switch 100 can participate inlogical tunnel end point 210. The virtual IP address can be assigned tothe member switches participating in logical tunnel end point 210. Ifany of these member switches receive the packet, that member switchconsiders the packet to be destined to the local switch and decapsulatesthe encapsulation header. In this way, fabric switch 100 can operate asa single logical tunnel end point for the tunnels established vianetwork 200.

The virtual IP address associated with logical tunnel end point 210 alsoallows fabric switch 100 to operate as a single layer-3 gateway (e.g., agateway router). For example, end device 114 can use the virtual IPaddress as the gateway IP address for all its communication. As aresult, when end device 114 initiates any communication, end device 114can issue an Address Resolution Protocol (ARP) query. When the queryreaches a switch participating in logical tunnel end point 210, theswitch responds with an ARP reply comprising a virtual MAC addressmapped to the virtual IP address. A switch participating in logicaltunnel end point 210 can maintain such mapping in a local storagedevice.

Upon receiving the reply, end device 114 uses the virtual MAC address asthe destination MAC address for its subsequent communication. In someembodiments, one of the switches participating in logical tunnel endpoint 210 can be elected to respond to a ARP query for the virtual IPaddress. All other switches forward ARP query to the elected switch. Ifthe elected with fails, another switch can be elected. In someembodiments, the switch with the highest (or lowest) switch identifiervalue is elected for responding to ARP queries for the virtual IPaddress.

If fabric switch 100 has a full mesh of tunnels for a respective memberswitch, fabric switch 100 does not need to maintain a separate routingprotocol for forwarding via the tunnels. For example, since switch 103has tunnel 218 with switch 105, a packet encapsulated in a tunnel headerwith the switch identifiers of switches 103 and 105 as source anddestination addresses, respectively, can be forwarded based on routinginformation of fabric switch 100. Switch 104 can receive such a packet,checks its local forwarding information, and forward the packet toswitch 105. As a result, packet forwarding between any two memberswitches can entirely based on tunnel encapsulation. An intermediateswitch, such as switch 104, does not need to decapsulate the tunnelencapsulation header.

However, a full mesh of tunnels can lead to a large number of tunnels.For example, with five member switches of fabric switch 100, the numberof tunnels is twenty five. Furthermore, for broadcast, unknown unicast,and multicast (BUM) traffic, a packet is replicated for a respectivetunnel. In some embodiments, tunnels in fabric switch 100 follow thephysical topology of fabric switch 100. For example, switch 103 does notestablish tunnel 218 with switch 105 since switch 103 is not directlycoupled with switch 105. As a result, a packet from switch 103 to switch105 is forwarded via tunnel 216 to switch 104. Upon receiving thepacket, switch 104 decapsulates the encapsulation header and examinesthe inner MAC address to determine switch 105 to be the egress switchfor the packet. Switch 104 then re-encapsulates the packet in anothertunnel header associated with tunnel 222 with the switch identifiers ofswitches 104 and 105 as the source and destination addresses,respectively.

This also allows fabric switch 100 to have a distribution tree oftunnels for the distribution of BUM traffic. For example, if switch 101is the root switch of the tree, upon receiving a packet belonging to BUMtraffic, switch 103 forwards the packet via tunnel 212 to switch 101.Switch 101 receives the packet, decapsulates the tunnel encapsulationheader, and examines the inner MAC address to determine the packet to bein BUM traffic. Hence, switch 101 uses the distribution tree to forwardthe packet. Switch 101 re-encapsulates the packet in another tunnelheader associated with the tree with the switch identifier of switch 101as the source address, and a multicast address associated with the treeas the destination address.

If a switch is in a logical unit, all switches of that logical unit canoperate as a virtual single tunnel end point in fabric switch 100. Forexample, logical unit 162 can have a single virtual IP address, whichcan be used as the tunnel end point for logical unit 162. Switches 103and 104 can initiate or terminate forwarding of a packet encapsulated ina tunnel encapsulation using that virtual IP address. In someembodiments, the scope of such encapsulation and termination is limitedwithin fabric switch 100 (e.g., the tunnel encapsulation header does notleave fabric switch 100). A respective switch of the logical unit canindividually operate as a tunnel end point as well. For example, bothswitches 103 and 104 can have their respective tunnels even though theyare in logical unit 162.

In some embodiments, a respective member switch periodically determineswhether a tunnel is operational. For example, a switch can useBi-directional Forwarding Detection (BFD) in the tunnels for detectionof unavailability associated with a link, link aggregation, or switch.Upon detecting the unavailability, the switch generates a notificationmessage, which indicates the type of unavailability, and sends thenotification message to other member switches. Since such unavailabilitychanges the adjacency of switches, the internal routing protocol of arespective member switch re-computes the paths in fabric switch 100.

FIG. 2B illustrates an exemplary tunnel encapsulation header for anIP-based fabric switch, in accordance with an embodiment of the presentinvention. A tunnel encapsulation header can include a tunnel header 260in addition to outer IP and outer Ethernet headers. Tunnel header 260can include a tunnel identifier 266, which identifies the correspondingtunnel. If the tunnel encapsulation in fabric switch 100 is based onVXLAN, tunnel header 260 can be an enhanced VXLAN header and tunnelidentifier 266 can be a 24-bit long VXLAN Network Identifier (VNI).

In a VXLAN header, before the VNI, 32 bits are reserved for additionalusage. In some embodiments, these 32 bits are used to represent alearning label 262 and a forwarding label 264, 16 bits each, in tunnelheader 260. The 8 bits after the VNI can remain as reserved bits 268.These enhancements allows fabric switch 100 to support but virtual linkaggregation groups, as described in conjunction with FIG. 5.Furthermore, the tunnels in fabric switch 100 can be agnostic tounderlying topology.

Network Extensions

In some embodiments, fabric switch 100 participates in asoftware-defined network (SDN). FIG. 3A illustrates an exemplaryIP-based fabric switch participating in a software defined network, inaccordance with an embodiment of the present invention. In this example,fabric switch 100 can be a heterogeneous software-defined network, whichcan include one or more switches capable of processing rules andconfigurations provided by a controller (such as those defined usingOpenFlow). These one or more switches can be referred to assoftware-definable switches. A controller 310 is logically coupled to arespective software-definable switch in fabric switch 100 via a network300 (e.g., a layer-2 or layer-3 network). Controller 310 can bephysically coupled to a subset of the switches.

One or more services supported by fabric switch 100 can be configuredfrom controller 310. Controller 310 can view fabric switch 100 as alogical chassis and provide configuration for the logical chassis. Uponreceiving such configuration from controller 310, the receiving switchcan distribute the configuration information to the member switches. Forexample, controller 310 can provide a set of port profiles for fabricswitch 100. Upon receiving the port profiles from controller 310, thereceiving member switch distributes the port profiles to other memberswitches. A port profile includes one or more MAC address of enddevices, and specifies a set of configurations for a port (e.g., QoS,VLAN, and security configurations). If a switch detects a MAC address ofa packet in a port profile, the switch applies the configurations ofthat port profile to the ingress port (and/or egress port) of thepacket.

Similarly, fabric switch 100 can receive internal routing information(e.g., paths between member switches) from controller 310. This allows amember switch to establish a tunnel and forward traffic via the tunnelto another member switch without running a routing protocol withinfabric switch 100. Controller 310 can also configure a global VLAN infabric switch 100. A global VLAN is a virtualized network in fabricswitch 100 and corresponds to a customer VLAN. A global VLAN identifiercan be 24 bits long and its scope can be limited within fabric switch100. Controller 310 can provide a mapping between the global VLAN andthe customer VLAN.

FIG. 3B illustrates exemplary Fibre Channel (FC) gateways in an IP-basedfabric switch, in accordance with an embodiment of the presentinvention. In this example, switches 101 and 102 can form an FC gateway360, and are coupled to FC storage area network (SAN) 350. In someembodiments, network 350 is an FC fabric and includes FC router 352. Oneor more target storage devices can be coupled to FC router 352. FCfabric 350 is dedicated to provide access to data blocks from thetargets.

Switch 101 and 102 can present FC router 352 as virtual switch 352 toswitches 103, 104, and 105. In some embodiments, mapped identifiers infabric switch 100 are in the same format as the domain identifier of FCrouters 352. Switches 101 and 102 advertise the domain identifier of FCrouter 352 as the mapped (or switch) identifier of virtual switch 352.In this way, switches 101 and 102 can forward FC over Ethernet (FCoE)traffic from an end device (e.g., end device 114) to an FC domain,thereby extending the domain of network 150 to the domain of fabricswitch 100. As a result, a single routing protocol instance in arespective switch in fabric switch 100 can make routing decisions fortunnel-encapsulated Fibre Channel or non-Fibre Channel packets.

Fabric Formation

FIG. 4 presents a flowchart illustrating the fabric-formation process ofa member switch in an IP-based fabric switch, in accordance with anembodiment of the present invention. During operation, the switchobtains a switch identifier (e.g., an IP address) for the local switch(operation 402). The switch can obtain this switch identifier from auser or an identifier allocation service (e.g., DHCP). The switch thenobtains a mapped identifier for the local switch (operation 404). Theswitch can also generate the mapped identifier based on the switchidentifier or based on an indexing service. The switch then discoversthe neighbor switches based on a link discovery protocol (e.g., LLDP)(operation 406).

The switch establishes link aggregation for the links coupling the sameneighbor switch (operation 408). In the example in FIG. 1A, switch 101forms a link aggregation 130 for the links coupling switch 103. Theswitch establishes point-to-point communication with a respectiveneighbor switch over inter-switch link using unnumbered layer-3interface (operation 410). The switch determines adjacency (e.g., paths)within the fabric switch using a routing protocol (e.g., a variation ofOSPF) (operation 412). The switch then establishes tunnels with othermember switches based on mesh preference of the fabric switch (operation414). For example, if the mesh preference indicates a full mesh, theswitch establishes a tunnel with a respective other member switch. Onthe other hand, if the mesh preference indicates a partial mesh, theswitch establishes a tunnel with a subset of other member switches(e.g., directly coupled member switches).

In some embodiments, the switch configures the switch identifier (e.g.,the IP address) as the domain identifier for the local switch (operation416). This facilitates additional compatibility for the switch tooperate in both IP-based and domain identifier based fabric switch. Theswitch then checks whether the switch supports operating in a softwaredefined network (operation 418). If so, the switch obtainsservice/feature configuration from a controller of the software definednetwork (operation 420), as described in conjunction with FIG. 3A.Otherwise, the switch initiates distributed service/featureconfiguration (operation 422). For example, the switch can initiatesharing of configuration information of a respective member switch, asdescribed in conjunction with FIG. 1C.

Virtual Link Aggregation

FIG. 5 illustrates an exemplary virtual link aggregation group in anIP-based fabric switch, in accordance with an embodiment of the presentinvention. As illustrated in FIG. 5, end device 512 and 516 are bothdual-homed and coupled to switches 103 and 104. The goal is to allow adual-homed end device to use physical links to at least two separateswitches as a single, logical aggregate link, with the same media accesscontrol (MAC) address. Such a configuration would achieve trueredundancy and facilitate fast protection switching. A link aggregationgroup coupling a device to at least two other devices can be referred toas a virtual link aggregation group.

Switches 103 and 104 are configured to operate in a special “trunked”mode for end devices 512 and 514, thereby forming a virtual linkaggregation group 530. End devices 514 and 514 view switches 103 and 104as a common virtual switch 502, with a corresponding virtual switchidentifier. Dual-homed end devices 512 and 514 are considered to belogically coupled to virtual switch 502 via logical links (denoted bydotted lines). Virtual switch 502 is considered to be logically coupledto both switches 103 and 104, optionally with zero-cost links(represented by dashed lines). Other switches may view end devices 512and 514 to be coupled with virtual switch 502.

Among the links in a link trunk, one link is selected to be a primarylink. For example, the primary link for end device 512 can be the linkto switch 103. Switches participating in a virtual link aggregationgroup and forming a virtual switch are referred to as “partnerswitches.” Operation of virtual switches for multi-homed end devices isspecified in U.S. patent application Ser. No. 12/725,249, AttorneyDocket No. BRCD-112-0439US, entitled “Redundant Host Connection in aRouted Network,” the disclosure of which is incorporated herein in itsentirety.

In a typical layer-3 redundancy protocol (e.g., Virtual RouterRedundancy Protocol (VRRP)), a dual-homed end device should supportlayer-3 communication. On the other hand, an end device coupling avirtual link aggregation group can be in layer-2. In the example in FIG.5, end device 514 can be a layer-2 networking device (e.g., an Ethernetswitch) coupling another end device 516. Since switches 103 and 104 canbe in logical unit 162, virtual link aggregation group 530 canfacilitate high availability within logical unit 162 for end devices 512and 514. Furthermore, switches 103 and 104 can initiate and terminatetunnel encapsulation based on the virtual switch identifier of virtualswitch 502. Upon receiving a packet from end device 512, switch 103 (orswitch 104) encapsulates the packet in a tunnel encapsulation header andincludes an identifier for virtual link aggregation group 530 in thetunnel encapsulation header, as described in conjunction with FIG. 2B.Furthermore, switch 103 can assign the virtual switch identifier ofvirtual switch 502 as the source address in the tunnel encapsulationheader.

Exemplary Switch

FIG. 6 illustrates an exemplary member switch in an IP-based fabricswitch, in accordance with an embodiment of the present invention. Inthis example, a switch 600 includes a number of communication ports 602,a packet processor 610, a logical chassis apparatus 630, and a storagedevice 650. In some embodiments, packet processor 610 adds anencapsulation header to a packet.

In some embodiments, logical chassis apparatus 630 maintains amembership in a fabric switch, which can represent itself as a logicalchassis. Logical chassis apparatus 630 maintains a configurationdatabase in storage device 650 that maintains the configuration state ofa respective switch within the logical chassis, as described inconjunction with FIG. 1C. Logical chassis apparatus 630 maintains thestate of the fabric switch and the logical chassis, which is used tojoin other switches. Under such a scenario, communication ports 602 caninclude inter-switch communication channels for communication within thefabric switch. This inter-switch communication channel can beimplemented via a regular communication port and based on any open orproprietary format (e.g., IP protocol).

Logical chassis apparatus 630 facilitates formation of the fabric switchrepresented as the logical chassis, as described in conjunction withFIG. 4. Logical chassis apparatus 630 also allows switch 600 toparticipate in the name service of the fabric switch. If switch 600 isin a logical unit, logical unit apparatus 632 maintains the logical unitfor switch 600, as described in conjunction with FIG. 1D. Tunnelmanagement apparatus 622 maintains tunnels with other member switches,as described in conjunction with FIG. 2A. In some embodiments, switch600 includes a tunnel end point apparatus 666, which operates thelogical chassis as an end point for an external tunnel (i.e., thetunnel's other end point is outside of the logical chassis). If switch600 supports software defined network, SDN apparatus 640 facilitatesconfiguration of switch 600 based on rules and configurations from acontroller, as described in conjunction with FIG. 3A. Link aggregationapparatus 624 supports virtual link aggregation groups in switch 600.

Note that the above-mentioned modules and apparatuses can be implementedin hardware as well as in software. In one embodiment, these modules andapparatuses can be embodied in computer-executable instructions storedin a memory which is coupled to one or more processors in switch 600.When executed, these instructions cause the processor(s) to perform theaforementioned functions.

In summary, embodiments of the present invention provide a switch and amethod for facilitating an IP-based logical chassis. In one embodiment,the switch includes a logical chassis apparatus and a tunnel apparatus.The logical chassis apparatus associates a logical chassis identifier ofa logical chassis with the switch and assigns an Internet Protocol (IP)address as switch identifier of the switch. The logical chassis includesa plurality of member switches and the switch is a member switch of thelogical chassis. The IP address uniquely identifies the switch in thelogical chassis. The tunnel apparatus establishes a tunnel with a remoteswitch in the logical chassis. An inter-switch packet from the switch isencapsulated in a tunnel header associated with the tunnel.

The methods and processes described herein can be embodied as codeand/or data, which can be stored in a computer-readable non-transitorystorage medium. When a computer system reads and executes the codeand/or data stored on the computer-readable non-transitory storagemedium, the computer system performs the methods and processes embodiedas data structures and code and stored within the medium.

The methods and processes described herein can be executed by and/orincluded in hardware modules or apparatus. These modules or apparatusmay include, but are not limited to, an application-specific integratedcircuit (ASIC) chip, a field-programmable gate array (FPGA), a dedicatedor shared processor that executes a particular software module or apiece of code at a particular time, and/or other programmable-logicdevices now known or later developed. When the hardware modules orapparatus are activated, they perform the methods and processes includedwithin them.

The foregoing descriptions of embodiments of the present invention havebeen presented only for purposes of illustration and description. Theyare not intended to be exhaustive or to limit this disclosure.Accordingly, many modifications and variations will be apparent topractitioners skilled in the art. The scope of the present invention isdefined by the appended claims.

What is claimed is:
 1. A switch, comprising: a logical chassis apparatusconfigured to: associate a logical chassis identifier of a logicalchassis with the switch, wherein the logical chassis includes aplurality of member switches, and wherein the switch is a member switchof the logical chassis; and assign an Internet Protocol (IP) address asswitch identifier of the switch, wherein the IP address uniquelyidentify the switch in the fabric switch; and a tunnel apparatusconfigured to establish a tunnel with a remote switch in the logicalchassis, wherein an inter-switch packet from the switch is encapsulatedin a tunnel header associated with the tunnel.
 2. The switch of claim 1,wherein the logical chassis apparatus is further configured to maintaina mapped identifier assigned to the switch, wherein the mappedidentifier is an index for the switch in the logical chassis.
 3. Theswitch of claim 2, wherein a port of the switch is identified by a portidentifier uniquely identifying the port in the logical chassis, andwherein the port identifier includes the mapped identifier.
 4. Theswitch of claim 1, wherein the logical chassis apparatus is furtherconfigured to determine adjacency in the logical chassis by running arouting protocol.
 5. The switch of claim 1, wherein the logical chassisapparatus is configured to operate the logical chassis as a singlemanageable entity for provisioning, control, or both.
 6. The switch ofclaim 5, wherein the logical chassis apparatus is further configured tomanage the logical chassis based on one or more of: a command lineinterface (CLI); a Network Configuration Protocol (NETCONF); andRESTCONF.
 7. The switch of claim 1, wherein the switch is in a logicalunit, wherein the logical unit is a building unit of a logical chassis.8. The switch of claim 7, wherein the logical unit includes a secondswitch, and wherein the switch and the second switch operate as tunnelend points for the tunnel.
 9. The switch of claim 1, further comprisinga logical tunnel end point apparatus configured to operate the logicalchassis as an end point for an external tunnel, wherein the other endpoint of the external tunnel is outside of the logical chassis.
 10. Theswitch of claim 1, further comprising a link aggregation apparatusconfigured to: identify a plurality of links coupled to a same neighborswitch; and operate the identified links as a link aggregation group,wherein the links in the link aggregation group operate as a singlelogical link.
 11. The switch of claim 1, wherein the switch is in asoftware defined network, and wherein the switch receives configurationinformation in an instruction from a controller of the software definednetwork.
 12. The switch of claim 1, wherein the logical chassisapparatus is further configured to establish a point-to-point connectionwith a neighbor switch using an unnumbered interface based on the IPaddress.
 13. The switch of claim 1, wherein the logical chassisapparatus is further configured to discover a neighbor switch based on alink discovery protocol.